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ABSTRACT 

This paper reviews 18 microcomputer software item 
banks on an individual basis. Software Pronrams reviewed are AIMS 
(Academic Instructional Measurement System); CREATE-A-TEST; Exam 
Builder; Exams and Examiner; MicroCAT; Multiple Choice Files; P.D.Q. 
Builder; Quiz Rite; Teacher Create Series (5 programs); TAP (Testing 
Authoring Program); TestBank Test Rite; Testmaster; Testmaster 
Series; Tests, Caicreate, Caitake; Tests Made Easy; TestWorks; and 
the Sage. Each review devotes one to three paragraphs to the 
following areas: (1) general program description; (2) item bank; (3; 
item bank maintenance; (4) test assembly; (5) test administration and 
scoring; (6) student recordkeeping; and (7) test use. Two tables 
summarize the reviews and allow for software comparisons. Table 1 
provides the program title; the vendor name and address; the computer 
used and major features; and list price of software. Table 2 gives a 
brief overview of the reviews in chart form. (LMO) 
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Table 1 

SUMMARY OP ITEM BANKING PROGRAMS REVIEWED 



Program 
Title 



Vendor 



Major List 
Computer Features Price 



CREATE -A Cross Educational Software 
A-TEST 1802 N. Trenton 

Ruston, IA 71270 



Apple Item Bank $ 90 

Maintenance $ 50/item 
Test Assembly bank 



Multi pie Compu-ta tions 

Choice P.O. Box 

Files Troy, MI 48099 



Apple Maintenance $ 30 
Ata r i Admin is tr a t ion 



P.D.Q. Micro Power and Light Company Apple 
Builder 12820 Hillcrest Road #219 
Dallas, TX 75230 



Maintenance 
Test assembly 
Administration 



$ 45 



Quiz Class 1 Systems 

Rite 17909 Maple Street 

Lansing, JL 60438 



Apple Maintenance $ 90 
TRS-80 Test Assembly 
IBM- PC 



Test Class 1 Systems 

Rite 17909 Maple Street 

Lansing, IL 60438 



Apple Maintenance $139 
TRS-80 Test assembly 
IBM-PC 



Test- Midwest Software 
master P.O. Box 214 

Farmington, MI 48024 



Apple Maintenance J 35 
CBM Test assembly 

Commodore 



Test- D.B.C. Computing IBM-FC 
master 5307 Lynnwood Drive Apple 
Series West Lafayette, IN 47906 TRS-80 



Item bank 



$ 20/ item 
bank 



Tests * EduSys terns 
Caicreat 3224 Lakeland Drive 
Caitake Berrien Springs, MI 



49103 



IBM PC Maintenance $ 75 Tests 
Test assembly $100 All 
Administration thr ye 



The Jagdstaffel Software 

Sage 645 Brer.da Lee Drive 

San Jose, CA 95123 



Apple Maintenance $ 62 
Administration 



Table 1 — Cont'd 
SUMMARY OF ITEM BANKING PROGRAMS REVIEWED 



Program 
Title 



Vendor 



Major List 
Computer Features Price 



AIMS 



Ex am 
Builder 



Charles Merrill Pub. Co. 
Test Division 
1300 Alum Creek Dr. 
Box 508 

Columbus, OH 43218 



A. U. Software 
P.O. Box 597 
ColleyvUle, TX 



76034 



IBM PC- XT Maintenance $900 min. 

Test Assembly (with 
Item Bank items) 



7RS-80 Test Assembly $100 
Commodore Maintenance 
Apple 
IBM PC 



Exams and Mf.crosof tware Services 
Examiner P.O. Box 776 

Harrisonburg , VA 22801 

MicroCAT Assessment Systems Corp. 
2233 University Ave. 
Suite 310 

St. Paul, MK 55114 

TAP Addison-Wesley Pub. Co. 

Medical-Nursing Division 
2727 Sand Hill Road 
Menlo Park, GA 94025 

Educational Courseware 
67A Willard St. 
Hartford, CT 06105 



Teacher 
Create 
Series 
(5 Programs) 



TestBank Holt, Rhinehart, 6 Winston 
383 Madison Ave. 
New York, NY 10017 

Tests Campu- tat ions 
Made P.O. Box 502 

Easy Troy, MI 48099 

TestWorks Milliken Publishing Co. 
1100 Research Blvd. 
P.O. Box 21579 
St. Louis, MO 63132 



TRS-80 Maintenance $ 70 each 
Administration $130 both 



IBM PC Test Assembly $975 

Maintenance and more 
Administration 



IBM PC Item Bank $125 
Maintenance 
Test Assembly 
Administration 

Apple Administration NA 
Maintenance 



Apple Item Bank NA 
Maintenance 
Test Assembly 

Apple Test Assembly $ 30 
Maintenance 



Apple Maintenance $250 
Test Assembly 
Administration 



25 



AIMS 



ItM Bank Description 

Nay purchase item banks X 

Use various response formats X 

Link text passage to items X 

Store classification scheme X 

Allow »ore than 4 lines for X 
item « tea 

Itea Bank Maintenance 

Create and edit iteas X 

Use full-screen editing X 
Store usage history 
Calculate itea statistics 

Test Asseably 

Select by itea number X 

Select randomly X 

Select by classification X 

Print multiple forms X 

Support special characters X 



Administration and Scoring 
Administer test on-line 
Support scanner 
Compute totals on test 
Determine objective mastery 

Student Recordkeeping 

Use objectives approach 
Use grade book approach 



Use Factors 

Comprehensive manual HA 

Easy to use NA 

Reasonable performance NA 



Table 2 



Create-a- Exam Exams* MlcroCAT Multiple P. D. Q- Quls Rite TAP The Sage 
Test Builder Examiner Choice Builder 

Piles 



X 
X 



X 
X 



x 
x 
x 



x 
x 
x 
x 
x 



X 
X 



X 
X 



X 
X 
X 



X 
X 



X 
X 
X 



X 
X 

X 



X 
X 



X 

X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 
X 



NA 

X 
X 



X 
X 
X 
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Table 2 ~ Cont'd 



Teacher 
Create 



Test Bank Test Rite Tests 



Tests Made 
Easy 



Test- Tes toaster Testtlorks 
iter Series 
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Iteai Bank Description 

Nay purchase item hanks 
Use various response formats 
Link text passage to items 
Store classification scheme 
Allow wore than 4 linee for 
item steal 

Item Bank Maintenance 

Create and edit items 
Use full-screen editing 
Store usage history 
Calculate item statistics 

Test Assembly 

Select by item number 
Select randomly 
Select by classification 
Print multiple forms 
Support special characters 

Administration and Scoring 
Administer test on* line 
Support scanner 
Compute totals on test 
Determine objective mastery 

Student Recordkeeping 

Itoe objectives approach 
Use grade book approach 

Use Pactors 

Comprehensive manual 
Easy to use 

Reasonable performance 
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X 
X 
X 
X 
X 



X 
X 



X 
X 
X 
X 
X 



X 
X 



X 
X 
X 



X 
X 



X 
X 



X 
X 
X 



X 
X 

X 
X 



X 
X 



X 
X 
X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 



X 
X 
X 



NA 
NA 



X 
X 
X 
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PRODUCT: 



AIMS: Academic Instructional Measurement System 



VENDOR: 



Charles E. Merrill Publishing Co. 

Test Division 

1300 Alum Creek Drive 

Box 508 

Columbus, OH 43216 



GENERAL DESCRIPTION 

AIMS is an item bank maintenance program which comes with any of four 
Charles Merrill item banks for primary and secondary educators. The principle 
use of AIMS is for classroom testing with the provided items or the teacher's 
own items. Since the items provided with AIMS are norm and criterion 
referenced, tests can be constructed for diagnosis or minimum competency. 

In order to use this package, an IBM- PC XT or AT or compatible computer 
with at least 10 MB hard disk is required. An Epson PX80 printer is needed in 
order zj print special characters. 

The program sells for $600 and must be purchased with an item bank. Item 
banks sell for between $300 and $2,300. Items may be purchased separately 
(printed cop/) and cover reading/language arts and mathematics for grades 1-8 
or 9-12. The complete system costs $3,600. A test development guide is also 
available for $40; it is a complete guide for developing norm or criterion 
referenced tests for classrooms, schools, or districts. 

The actual program apd manual were not available for review from the 
publisher. This product discription is based on promotional materials and 
conversations with a Charles Merrill programer. 



IT34 BANK DESCRIPTION 

Item storage can be in any format desired. Questions and answers are 
stored together, allowing the user to store f ill-in-the-blank and essay 
questions as well as multiple-choice. Reading passages may be stored as test 
items and can be of any length. 

Items are stored in six "layers": the text, any available item 
statistics, the objective which the item measures, a classification scheme , 
associated passages, and any directions. All layers are linked together. 
Printed artwork comes with each item bank and is linked by item code to the 
correct items. Purchased items include statistics, objectives, and 
classifications. 



ITEM BANK MAINTENANCE 

A full-screen editor available through another distributor is incorporated 
into AIMS. This program provides complete item tntry and editing 
capabilities. The instructor can use upper and lower case, underlining, 
superscripts, subscripts, and other special characters not normally available 
with item bank editors. 
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TEST ASSEMBLY 

There are several options available for assemblying tests. The user can 
predetermine which items will be used and select on those items. A second 
method would be to choose objectives to be measured and allow the computer to 
select randomly from within objectives. Thirdly, tests can be constructed 
with specific statistical parameters by selecting items based on their 
difficulty, discr imation, and item-total correlations. All item statistics 
for user-developed items must be calculated elsewhere, but may be stored with 
each item. 



TEST ADMINISTRATION AND SCORING 

Printed tests are camera ready and can be presented by photocopying for a 
single class or professional printing for a school or district. 

There is no provision for on-line testing or computer scoring. 



STUD BIT RECORDKEEPING 

There is no provision for recording student scores. 



OSE 

Neither the manual nor the program disk were made available for review. 
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PRODUCT: 



CREATE -A -TEST 



AUTHOR: Cr eate-a-Test Company 
80 Tilley Drive 
Scarborough, Ontario MIC 2G4 

VENDOR: Cross cations Software 
1802 N. Trenton 
Ruston, LA 71270 



GENERAL DESCRIPTION 



Create-A-Test is a program for creating and maintaining test item files 
and creating classroom tests or quizzes. An Apple II plus with 48K memory and 
DOS 3.3 is required. One disk drive is needed, but two can be accommodated. 
It is available for about $90. Previously developed item banks in 
secondary-level subjects of science and social studies are available for about 
$50 each or five for $200. Each disk has 400 tc 500 questions. The program 
plus four item disks can be purchased for $200. 

Three copies of the program and one blank disk are provided with the 
manual in a padded three-ring binder. Storage folders within the binder can 
hold eight disks. 

Most printers installed for the Apple computer are supported. 

I TO! BMW DESCRIPTION 

Items of any type can be stored, as well as reading passages or 
instructions. Items are stoied in files called "tests" which can hold up to 
32 questions each. Each disk holds up to 14 files. Stems are limited to 
about 240 spaces, but items may be linked together and need not have response 
options. This allows the user to file a reading passage, item instructions, 
or especially long questions. A long question can be continued from the disk 
space assigned to one item to the space assigned * > the next item. When 
printed, these two "items" will be printed as one. In addition, items may be 
given an identifying mark indicating that they pertain to the previous item 
(question, passage, or instruction). 

No classification scheme is linked directly to each item, but the 
developers have allowed for item storage by classification. Each item storage 
disk may be numbered. Thus users can define their own classification code for 
each disk. In addition, items may be numbered at the user's discretion. This 
can provide a further classification scheme. A master list of sub-topics and 
the item numbers included can be made. Items added to an item bank will 
automatically go at the end. This order of entry can be overridden in order 
to place a new item in the proper numbering sequence. 
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ITEM BANK MAINTENANCE 

A full-screen editor with commands similar to the popular package Wordstar 
is provided* This is easy to learn and easy to use. Control commands are 
listed at the bottom of the screen while in edit mode. Superscripts and 
subscripts are supported, as well as lower case. 

Item banks are available for purchase in eight secondary subjects 
(Biology, Physical Science, Physici, Chemistry, Botany, Government, History, 
and Geography}. These provide a total of between 400 \nd 6,000 items in each 
subject. 



TEST ASSEMBLY 

Test items are selected by specifying the number of the question. Items 
cannot be selected randomly. Selection by classification is done by the 
instructor by selecting from the user-defined storage system. Dhlike some 
other packages, items are not shown on the screen during the selection 
process. This greatly speeds selection, but the user must have a hard copy of 
the item bank in order to know which items to select. 



TEST ADMINISTRATION AND STORING 

Tests are administered only by printrd copy. No scoring or item analysis 
is available. 



STUDENT RECORDKEEPING 

There is no provision for student recordkeeping. 



USE 

The manual is complete and easy to follow. Sections are clearly titled 
and there are many diagrams and examples. In fact, most of the 39 pages are 
examples of printouts or what the screen looks like. Create-A-Test provides 
great versitility in item storage which adds some complexity to its use. 
General operations go quickly once the system is understood. 
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PRODUCT: Exam Builder 

VENDOR: A. U. Software 
P.O. Box 597 
Colleyville, TX 76034 



GQIERAL DESCRIPTION 

Exam Builder is a program for creating and maintaining test item files and 
creating tests for classroom use. Five files of 150 questions can be stored 
on one disk. There is no limit to the number of disks used for storage. Only 
one disk drive is needed, but a printer must be installed for the program to 
operate. 

Nearly any computer available in schools can be used. Versions of the 
package are available for TRS-80, Comnicdore-64, Apple II (+, e, and c) and 
IBM personal computers. The package sells for about $100. The IBM version 
was used for this review. 



ITEM BANK DESCRIPTION 

Items can only be of multiple-choice or true-false format. Each question 
may have two to five responses. Items are stored in "subject" files of up to 
150 questions each. Five files can be put on each disk, but there is no limit 
to the number of disks that can be used. Each question may have up to 244 
characters in the stem and a combined total of 244 characters in all 
responses. If the user exceeds the 244 character limit, the program does not 
adequately trap the error. 



ITH4 BANK MAINTENANCE 

While entering items, th^ ler may use all the editing keys available with 
their .cmputer (e.g., IBM r cursor control and delete keys) before 

pressing the Enter I key. " a stem or response option has been entered, 
it cannot be edited and mubw oe entirely retyped. The limited editing ability 
of the pr- iram is not mentioned in the manual. 

The program require that a printer must be attached and on while the 
program is being used, &n awkward feature. Each item is printed as it is 
entered. This does provide a written copy of each file, but at the expense of 
time and paper. An option is provided of also printing out any file or 
portion of a file at a later time. 
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TEST ASSEMBLY 

Test items may all be chosen from one file or from any number of files. 
Within a file, items may be chosen randomly or at the user's discretion. 
Before choosing items for the test, the teacher ca write a heading of up to 
five lines. Each line is automatically centered before printing. 

A separate key may be printed after a test or item file. The printing of 
this key is password protected, as is the changing of the password. 

9 
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TEST ADMINISTRATION AND SCORING 

Tests are administered only by printed copy. No scoring or item analysis 
is available. 



STUD BIT RECORDKEEPING 

There is no provision for student recordkeeping. 



USE 

The manual is fairly complete and fairly easy to follow. Some basic 
information about the program should be included at the beginning and is not. 
Most of this needed information is in a "Hints and Notes" section on the last 
page. Much time was wasted before reading that page. 

Most decisions required of the user need only a yes or no response. The 
proper response is "Y" or "N". Any other response, including "y" or "n", 
produces unpredictable results — often something other than what the user 
wants; sometimes resulting in a need to reboot, which can erase any newly 
entered items. 
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PRODUCTS: Exams and Examiner 

AUTHOR: Michael Stoioff 

VENDOR: Micr ©software Services 
P. 0, Box 776 
Harrisonburg, VA 22801 



GB9ERAL DESCRIPTION 

Exams is a item storage and test construction program. Examiner is an 
on-line testing program using t^sts produced by Exams. Banks of 
multiple-choice items can be produced from which tests or quizes may be 
constructed. 

These packages require a 7SS-80 Model 3 or 4. Each package sells for 
about $70, but both can be purchased for $130. Anyone wishing to examine the 
package may buy the manual separately for $10. 



ITEM BANK DESCRIPTION 

Items are stored in a ml ti pie-choice format with five response options. 
Questions can be extremely long --up to 17 lines. Nearly 260 questions may be 
stored in a single file. A "keyword" line is available on each item for 
listing classification titles for later selection. Questions may be copied 
from a storage file and placed in a test file with page neadings and test 
instructions included. 



ITEM BANK MAINTENANCE 

An editor is provided for entering, updating, and correcting items. It 
could be called a "full-line" editor in that editing may be done within any 
line. Changing more than one line is a little more difficult. Without the 
manual, use of the editor would be nearly impossible. The editor's menu is 
confusing and incomplete, as well as poorly structured. Pbr example, any time 
that an item is added or edited it must be specifically saved. The menu lists 
" Enter I to continue", but :tr iking the enter key abandons the changes and 
takes the user to the next menu choice. 



TEST ASSEMBLY 

Any file on the disk can be considered to be a test. If an item file 
contains more items than are wished for a test, the desired items may be 
selected and copied to a new file. Items may be selected by number, by 
reviewing the file, by random selection, or by keywords. Keyword selection 
involves the computer searching the file for a specific word or part of a 
word. The word may be chosen from the item or the "keyword" line. Thus tne 
user may search for any question containing a desired word or for any question 
stored with a desired classification title. The search may also be limited to 
just the keyword line. 
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TEST ADMINISTRATION AMD SCORING 



Exams prints out a blank answer sheet, a key, and the finished test* This 
copy of the test can be duplicated and used in the classroom. 

Examiner uses the test produced on the Exams disk and administers the test 
on-line. Immediate feedback is given to each answer. After each test, 
students are given their scores, the time they took, and the list of answers 
given and which ones were wrong. 



STUDENT RECORDKEEPING 

Examiner saves the information given to the student on disk for the 
teacher. Also recorded are the scores of all students in the class in a 
gradebook format and some simple statistics for each question— item difficulty 
and proportion of students selecting each response. Statistics must be 
computed elsewhere. 



USE 

A separate manual is provided with each program. This manual is well 
organized and comprehensive. In addition, the vendor is willing to give 
assistance over the telephone. The manual and phone assistance are 
necessary. Though Exams is menu driven, the structure and clarity of the 
menus are poor. Performing simple actions is difficult to do and often 
involves excessive time. Examiner, on the other hand, is easy to use and 
serves a useful purpose by storing student scores and item statistics. 



PRODUCT: MicroCAT 

VENDOR: Assessment Systems Corp. 

223: "-'^ersity Ave., Suite 310 
St. Pftux, MN 55114 



GENERAL DESCRIPTION 



MicroCAT is a highly sophisticated program for computerized adaptive 
testing. With creative use, this package could also be used for instruction, 
diagnosis, or mastery learning. Unfortunately, no provision is allowed for 
printing tests. Pour separate programs are available: the item and test 
development subsystem, the examiniation subsystem, the Item and test 
assessment subsystem, and the test management subsystem. Items can be created 
and stored, compiled into tests, and presented by computer at stand-alone 
stations oi in a network. The assessment subsystem computes test scores and 
item and test statistics for compute represented tests as well as data files 
created from paper and pencil tests. 

An IBM personal computer or similar computer with a colo. monitor is 
needed for MicroCAT. to printer is needed; nor can one be used. An advanced 
graphics editor is included to allow construction of color graphics to 
accompany or be incorporated into any item. If a mouse is available, it can 
be used, but is not necessary, though it may speed up the use of the graphics 
editor. When using the assessment subsystem for computing statistics on large 
files (such as a school district or large college lecture class), an 8087 
co-processor would greatly help. 

MicroCAT may be used in ml ti pie-user networks. Subroutines allow for 
monitoring test taking from a central proctor's command station. 

The item and test development subsystem comes with the examination 
subsystem and the user's manual for $975. If a network is to be used, the 
test management subsystem may be purchased for an additional $500. The item 
and test assessment subsystem may be purchased separately for $900. rhe full 
package costs about $2,400. 



ITEM BANK DESCRIPTION 

Items can be stored in two modes: color or black and white. The black 
and white mode is used strictly for text. Text in that mode can be up to 25 
lines of 80 characters each. In the color mode a full-function color craphics 
editor is available. 

Text may be entered in either mode. Ttext entered in color mode can be of 
any color desired. Text in either mode may also use up to 124 special 
characters. The user may create any character or symbol desired. These 
characters are created with a font generator. The font generator and the 
large number of characters available provides great flexibility. 
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The graphics editor in quite similar to MacPaint used on the Apple 
Macintosh. The graphics are impressive *nd can add immensely to a test. 
Unlike MacPaint, there is no "eraser". Mistakes can only be removed by 
backing through the file. This process can be quite time consuming (and 
mistake prone) when many graphic entries must be stepped through. 

Any "item" in the file can have any of a variety of uses. An "item" may 
be a test question, a reading passage, a picture, a chart or graph, or any 
kind of message or instruction. In order to provide full use for each kind of 
item, each item is stored with a large list of information. 

One important piece of stored information is the item identifier. This 
code consists of 1 to 6 letters and 1 to 3 numbers. It is primarily useful as 
a way to designate item clasif ications and to number items within 
class i f ica tion 8 • 

Items can be linked together. Each item can be stored with the identifier 
of an item to precede it or to follow, or both. This is most useful when 
items are to refer to a graph or passage. Items which follow another item can 
be presented on the same screen. For instance, a graph can be presented and a 
question may be superimposed over the graph. 

Tests can be made to test response time. If the teacher wishes to, items 
can be presented for a defined period of time. If the student does not answer 
in time, the item will be removed and the next one presented. This feature 
can also be used to present a reading passage for a specific length of time. 

Lastly, each item can be stored with up to three item parameters to be 
used primarily with adaptive testing using item response theory. The test 
manager may use these with the Rasch one-parameter mcdel, a two-parameter 
model, or the full three-parameter model to select items for a test. 



ITEM BANK MAINTENANCE 

Items can be added to the file or edited using the graphics editor at any 
time. The manual does not discuss the number of items which can be stored. 
It appears that about 175 items can be stored on a disk. This may vary 
depending on the percentage of graphs stored. Only one disk may be used for a 
given test. Using a hard disk for itsm storage would increase the pool size. 

Actually looking at a number of items can be difficult. Items cannot be 
printed. Dhlike some other item banking programs, the user cannot scroll 
through the item file. Each item must be accessed individually using its 
unique identifier code. Getting a list of valid codes is a two-step process 
which also cannot be printed (except with the "print screen" function of the 
IBM PC) . 

Though the program is generally menu-driven, there are no editing prompts 
in the text editing mode. This requires keeping the manual close at hand. 
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TEST ASSEMBLY 

Test assembly in MicroCAT is the most advanced of any of the programs 
reviewed. Mo other program reviewed used item response theory to produce 
tests. Six options are available for oonstucting/presenting tests: three 
types of conventional cest forms and three types of adaptive testing using 
item response theory. In addition, each item can be calibrated using either a 
three-parameter m&lel or the Rasch two-parameter model. 

MicroCAT uses the Minnesota Computerised Adaptive Testing Language (MCATL) 
to present its tests. Users are given a complete manual of this language in 
the user's manual. Those who am not adept at computer programing need not 
use this language. A compiler is included which allows the user to identify 
the test type and the items to be used and the computer writes the MCATL 
program. The MCATL program is used by the examination subsystem when the test 
is presented. 



TEST ADMINISTRATION AND SCORING 

Test administration is only on-line. No printed tests can be 
constructed. Tests can be presented on several computers at once or on one 
computer. One computer can give one test and stop or give tests 
consecutively. Presentation is coordinated with a different program disk than 
is used for editing or scoring, thus students cannot change items or answers. 

The examination disk stores a record of each student's responses to each 
question. A provision is included that allows a student to continue a test if 
they are interupted before completing the test (such as by a power failure). 
As student's take a test, their answers are stored in a data file. This data 
file can be accessed by the item and test assessment subsystem. 

The assessment subsystem not only can give total test and sub-scale scores 
for each student, it can provide a complete range of descriptive and analytic 
statistics including point-biserial correlations for each item and response, 
sub-scale intercorrelations, and KR-20. 

Lastly, the assessment subsystem can be used to determine validity scores 
for any test. A multiple regression program is built in which can be used to 
correlate tests, subscales, and items with a criterion score. 



STUDENT RECORDKEEPING 

Pbr each student, the number-correct score for each scale can be stored 
and printed out. If a minimum score is important (such as for competency 
testing), the MCATL program can be written to determine if the student 
attained the required level. A message pertaining to the students attainment 
of the minimum score is also stored. 
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USE 

At first glance, the 174-page manual for MicroCAT is rather overwhelming. 
Included in the manual is an introduction to laptive testing and a manual for 
use of the MCATL test construction program language. Actually, almost all 
actions within MicroCAT are driven by menus. The manual is easy to read and 
easy to follow. Dae to the complexity of this package, it is important to 
read the manual first and keep it handy until the commands are well learned. 
This is most true with the graphics editor where all commands are two- letter 
codes and there are no on-screen helps in the text mode. 

What this package dees is very impressive. The use of such advanced 
giaphics and the availability of such complex statistics are not in any other 
program reviewed here. On the other hand, the program is of limited use 
beyond computerized adaptive « sting. In addition, several problems were 
found in connection with item instruction. These problems are described 
below. 

When first cl eating an item, the user gives it a unique letter and number 
code. The letters define an item class. If an error is made when typing the 
class, that class will be filed onto the disk. The user can change that one 
item a identifier code, but the incorrect class code will continue to clutter 
the class code directory. There is no vay to remove it. 

The manual could be a little «ore explicit when it discusses the graphics 
editor, it seems to assume that the user has experienced similar programs. 
At one point it discusses how to store and execute segments, but dees not 
explain what a segment is. It also warns against improperly executing 
segments, but does not say how to avoid disaster by properly executing them: 
While reviewing the program, disaster did occur an* we could not correct it, 
despite many tries. 

When compiling a test, one graph would not be accepted by the compiler, 
apparently due to its size. Mo explanation is given in the manual and no 
warning is given in the program that graphics items can fit on the screen and 
still be unusable due to their complexity. 

A final set of major problems involves test assembly. When defining 
tests, at least three files must be created. These can be confusing, since 
their purpose is not made clear. If a mistake is made in naming or creating 
files, the mistake is saved on the disk. The only way to remove the mistake 
(or any outdated test file) is through the DOS delete/erase function. At one 
point users can get ahead of themselves and not have created needed files. 
This error is not trapped. To get out, the computer must be rebooted. 
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PRODUCT: Multiple Choice Files 

VENDOR: Carapu-ta t ions 
P.O. Box S02 
Troy, MI 48099 



GBtERAL DESCRIPTION 



Multiple Choice File simple item storage and retrieval system. 

Tests can be administered ,c printed from stored files, but items can not be 
individually selected from a file. Tests may printed, but the nin purpose of 
the program is to produce on-line testing sessions. Questions are presented 
randomly, with correct answers reinforced and feedback given for incorrect 
answers. 

The program runs on an Apple II or Atari 800/1200 and sells for about 
$30. The Apple version was reviewed. 

ITEM BANK DESCRIPTION 

Question files of up to 30 items can be entered and saved. Only 
multiple-choice items i be saved. Each question must have five possible 
response options. No commas, colons, or quotation marks may be used in the 
stem or responses. 

ITEM BANK MAINTENANCE 

Items may be added until the 30-item limit is reached. In order to edit 
an itera the entire item must be retyped. Charges are not automatically 
saved. After all editing is couplets the revised test file must be saved. 

TEST ASSEMBLY 

Items can not be selected from a question file, the test consists of all 
items in the file. If the test is printed, items are listed in the order 
entered. If on-line administration is used, icems are presented in random 
order. 



TEST ADMINISTRATION AND SCORING 

Printed tests are scored manually. An answer key is printed separately 
after the entire test is printed. 

When tests are administered on-line, items are presented in random order 
Correct Answers are rewarded and incorrect answers prompted with the correct 
response.. Any incorrectly answered question will be presented again. All 
questions are presented until answered correctly. 
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STUDENT RECORDKEEPING 

With on-line presentation a listing is provided of the name of the quiz, 
the student's name, the number of items presented, the nunfcer of items missed, 
and the percent correct. The percent is based on all presentations, thus a 
student may miss all ten questions of a ten-item quiz, get each question 
correct on the third try and end up with a score of 33 percent (ten right 
answers out of 30 presentations). There is no provision to determine the 
student's score based on number correct on first presentation. 

Student information listing is intended for use by the teacher to record 
the student's score. No permanent student record is kept on the disk. 



OS 13 

The program uses an easily followed menu format. A short, 6-page manual 
provides sufficient added explanation. Test files may be protected from 
editing, but students can take the Review option and see the answers before 
they are tested. As an on-line testing program, Multiple Choice Files is best 
used for practice and review or under close teacher supervision. 
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PRODUCT: P. D. Q. BUILDER 



VQ1 DOR: Micro Power and Light Company 
12820 Hillcrest Road #219 
Dallas, TX 75230 



GENERAL DESCRIPTION 



P. D. Q. Builder is a test or quiz construction program which allows easy 
construction of tests for paper or on-line presentation. The user can develop 
and maintain item banks for testing or review purposes. Tests can be produced 
on separate disks from the program disk — allowing on-line test presentation 
with assurance that computer wise students cannot change the program or the 
questions. 

The program is meant to expand on and replace the Adaptable Skeleton r 
another software product by this company. 

This package requires any Apple II computer with either one or two disk 
drives. It sells for about $45 and includes a comprehensive manual with one 
program disk and one sample data disk on which the items are stored. 



ITEM BANK DESCRIPTION 

Items in this syste* consist of the item stem and answers, as well as 
written feedback for use with on-line presentation. Items are stored in sets 
called "worksheets". Each worksheet can hold up to fifteen items. Items are 
not categorized r but each worksheet can be used to hold a specific category. 

Item stems are limited to 200 spaces for true/false and multiple-choice 
questions and 250 spaces for completion questions. This item length is longer 
than many packages reviewed. Up to 150 items nay be stored on a disk. 
Unlimited disks may be used when choosing items for a test. 



ITEM BANK MAINTENANCE 

As with all other aspects of this program, the user is guided through 
initial item entry by detailed, nested menus. When adding a new item, a 
series of underlines are shown on the screen that define the length allowed 
for the stem and response. This feature is helpful in that it gives fair 
warning when the limit may be exceeded, saving the necessity of re-entering 
the item if the stem is too long. 

Editing features of P. D. Q. Builder are limited. If a change in the stem 
is needed, the entire stem must be retyped. The same holds true for each 
response. At least the entire question does not need to be re-entered. 
Super- or sub-script are not available, but the user may highlight words or 
passages which appear as bold face when printed. 
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Bach item is automatically saved as it is completed or edited. This 
prevents accidental loss of items if the user forgets to save or if electrical 
power is interrupted. 



TEST ASSEMBLY 

When assembling a new test, the instructor may choose up to 10 items from 
up to 5 "worksheets" (item sets). Each worksheet nay be used as a sub-scsle. 
Items cun only be selected randomly from each worksheet. This can be a major 
problem in that if a worksheet is used as a way of categorizing items, the 
user must take all of the items in that category or else take "pot luck" on 
the items chosen. If specific items are needed, those and only those items 
must be in the worksheet. 

Items may be presented either in the order added to the test or in a 
random order. For paper tests the order can be rearranged to produce multiple 
forms of the test. 

Each test is placed on its own disk. This disk contains programs to allow 
for item selection, test presentation, and scoring. The item editing program 
is erased from the test disk. This feature allows the instructor to use a 
test disk for testing or review with the confidence that no student can change 
any of the stems or responses. 



TEST ADMINISTRATION, SCORING, AND RECORDKEEPING 

Test administration may be by standard paper and pencil presentation or by 
direct on-line interaction with the computer. Printing of items occurs 
directly from the item bank disk without the need to make a student disk. 
Entire "worksheets" or randomly selected items may be printed. A separate 
answer sheet is printed after the questions. 

On-line presentation can take two forms: testing or review. The 
difference being whether the instructor has the program save student records 
or not. If the student records are not saved, the student may take the test 
as often as they would like. After each question a short tutorial response is 
available which can be used to reward correct answers or guide incorrect 
answers, if student responses are saved, the student must re-run the testing 
program. The results of the second administration are also recorded. 

Records are saved in a password-protected file with each students 1 name, 
class period, date test was taken, orksheet Chamber assigned to the worksheet 
or "M" for multiple), number of items attempted, and number of correct 
responses. 



USE 

A 28-page manual accompanies the program. It gives a complete 
step-by-step description of how to do each activity. In addition, the user is 
thoroughly guided by the built-in menus in the program. The manual is 
necessary only for initial familiarization with the package and for 
clarification in the rare case that the menu may not be clear enough. The 
manual itself is clearly written and well organized. The reader can find 
needed information without having to read excessive pages of material. 
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PRODUCT: QUIZ RITE 

AUTHOR: Richard Van den Berg 

VENDOR: Class 1 Systems 

17909 Maple Street 
Lansing, IL 604 3 8 



GENERAL DESCRIPTION 

Quiz Rite is a program for creating and maintaining test item files , based 
on Test Rite by the same company. Items are stored in multiple-choice format, 
but may be printed in any format. Essay questions may also be stored. The 
package is good for regular classroom test construction. 

Quiz Rite uses the Apple II line of computers with 48K of memory, one or 
two disk drives, and virtually any printer. The version reviewed for this 
report was not complete and had no manual. Suggested retail price is $90. 
Versions are also available for TPS- 80 and IBM PC computers. 



ITEM BANK DESCRIPTION 

Like Test Rite, items are stored as an opei -ended question and potential 
responses, the correct response entered first. Up to five responses per item 
are allowed. Each item (including responses) can have up to 243 characters. 
There is no provision for storing a classification code. Essay questions are 
stored in a separate file. The user does not need to follow the required 
format convention, but nonconformity could result in incorrectly worded test 
questions, if care is not taken when selecting items. 



IT04 BANK MA IN TBI AN CE 

Items may only be added or removed. There is no full-screen editor. 
Editing is done by rewriting the stem or a response. 



TEST ASSEMBLY 

Quiz Rite selects items for a test in a less complicated manner than Test 
Rite. The user may choose specific items by passing through the file one item 
at a time and identifying the items to use. This routine may be left at any 
time without having to pass through the whole file. Items not specified will 
not be included. The program can also choose items by random selection. 

Before printing, the user may add any instructions that are to go on the 
test. The only additional information printed on the test are the date and 
the test name and number. 

Test item files are stored by both test name and course number, which can 
provide a kind of classification system. 
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TEST ADMINISTRATION AND SCORING 



Test administration is by printer only* The printed copy is not of the 
sane high quality as with Test Rite. Question sets nay be combined so that a 
long t*st using a variety of item kinds can be produced. An answer key is 
printed on a separate sheet. 

There is no provision for computer sooring. 



STUDENT RECORDKEEPING 

There is no provision for student recordkeeping. 



USE 

Quiz Rite is entirely menu driven. It is very easy to use* but what is 
gained in increased ease is lost in quality and flexibility. 

When choosing from menus $ the user is saved the extra keystroke of 
pressing "return" after each choice. Errors are generally caught by the 
program, but diagnostics are n t provided — you can only try again. One error 
while testing the program resulted in leaving the program and going back to 
DOS. This may have been corrected before the production version was finished. 
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PRODUCT: Teacher Create Series (Multiple Choice) 

VENDOR: Educational Courseware 
67A Millard St. 
Hartford, CT 06105 



GENERAL DESCRIPTION 

Teacher Create is a set of simple programs for creating and storing 
complete tests. Tests can be administeied by computer or be printed for 
clc s room presentation. Questions are presented in the order they were 
entered. Response options are presented randomly. With on-line presentation, 
immediate feedback is given. 

The programs run on any Apple computer using DOS 3.3. programs available 
*re: (1) Multiple-Choice, (2) Matching, (3) True and false, (4) Completion, 
and (5) Question/Answer Utility. The Multiple Choice disk was reviewed. 



ITEM BANK DESCRIPTION 

Test files of up to 100 items may be entered and saved. Tests can only be 
composed of the question type served by the specific program. With Multiple 
Choice each question must have five response options. "None of the above", 
"All of the above", and similar answers cannot be use<* since responses are 
shuffled with each presentation. 

The stem cannot exceed three screen lines. Each answer must fit onto one 
line. 



ITEM BANK MAINTENANCE 

The editor is quite simple. In order to corrrect the stem or one of the 
answers. The entire stem or answer must be retyped. Changes are automatically 
and inmediately saved. 

TEST ASSEMBLY 

Tests are stored in the order they are to be presented. Bach item file is 
a test file. The computer does not select from a l*rae item file to create a 
new test. 



TEST ADMINISTRATION AND SCORING 

Tests my be printed for classroom administration. Response options are 
printed in random order within quea ion, providing slightly different versions 
if printed more than once. Answers are printed on a separate page. 

When tests are administered on-line, correct answers are rewarded with 
musical sounds and a written phraxe such as "Wonderful". Incorrect answers 
are prompted with the correct response and a different musical sound. At the 
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end of the test, the student's score *3 presented on the screen. No record is 
kept of class scores. With immediate feedback and no recordkeeping, the 
on-line use of Teacher Create is belt for student practice and review. 



STUDENT RECORDKEEPING 

There is no provision for student recordkeeping. 



USE 

The program uses an easily followed menu format. An instructional session 
is included on the disk which basically repeats the information in the 
thre^-page manual. Teacher Create is so simple that no further information is 
needed to use it. Almost anyone could sit down and begin using it 
immediately. Test files are password protected to prevent unauthorized access 
by students, but the password cannot be changed. 

Using large files is very time consuming. Each time a test/file is 
accessed it must be loaded into memory. This loading takes about 3.S seconds 
per item. Thus, before taking or editing a 100-item test there could be a 
delay of up to five minutes. 
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PRODUCT: TAP: Test Authoring Program 



AUTHORS: Robert Slaughter, William Holzemer, 
Donald Chambers, and Nark Slichter 

VENDOR: Addison-Wesley Publishing 
Medical-Nursing Division 
2727 Sand Hill Road 
Nenlo Park, GA 94025 



GQJERAL DESCRIPTION 

TAP is an item banking and test authoring program for instructors or 
testing supervisors at any level. Items of any response format can be 
classified in up to seven catagories. Passages and graphs may be stored and 
linked to questions. 

An IBM (PC, PCAT, or PC/AT) computer with at least 152K of memory is 
required. Graphics can be used, if a compatible printer is available. An 
Apple version is under production. 

TAP is available for $125. This includes the program disk, a tutorial 
disk, the manual, and chree-ring binder. A 30-day moneyback guarantee is 
included, if after reviewing the tutorial, the purchaser decides not to keep 
TAP, a full refund is available — providing the sealed program disk has not 
been opened. 

Item banks are available which accompany four textbooks in nursing. These 
include between 500 and 1,000 items which can be added to and edited at the 
discretion of the nursing instructor. 



ITH4 BANK DESCRIPTION 

Items are entered into a space 24 lines long and 70 columns wide, with 
the editor feature, anything can be entered into this space. Generally this 
will consist of an item stem and responses. Essay and completion items may 
also be entered. Written passages, graphs, or graphics may also be stored and 
linked to as many as 20 items. In order to develop graphics and use special 
characters, the user must be familiar with the entry of ASCII codes with 
PC-DOS and have a printer capable of printing these special characters. 

The item editor is quite versatile. The one short-coming is that no 
wrap-around feature is included. A bell is sounded when text nears the end of 
the line, just as wicn « typewriter. If the end of the line is reached, the 
cursor will continue the word on the next line. Little effort is needed to 
edit such broken words. A second problem is that only one character can be 
inserted at a time before the program reverts back to overwrite mode. Thus, 
inserting a three-letter word takes six key strokes (pressing "insert" before 
each letter). 
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Items are stored with the date they were created, the correct answer (if a 
multiple-choice item), and up to six different classification areas: content, 
sub-classification, cognitive level, a user-defined scheme, item difficultv 
(.00 to 1.00), and item discrimination (.00 to 1.00). Only the content and 
sub-classification categories are required, specific labels are determined by 
the user. Excluding difficulty and discrimination, over 4,000 unique 
classification codes may be produced. 



ITEM BANK MAINTENANCE 

Items are entered and edited using the full-screen editing features 
described above. Only about 160 items may be stored on a disk. This is a 
major limitation. A 500-item bank would have to be stored on four disks, a 
1,000-item bank on seven disks. Multiple disks may not be used when 
constructing tests—making the large classification scheme of little practical 
use. All items on a single floppy disk should be in just a few content areas 
in order to maximize the pool of items for a test. Using a hard disk drive 
exclusively for item banking would avoid this problem. 

Each item disk has its own password and ID. This gives some protection to 
unauthorized access—unfortunately, if one disk is accessed and replaced with 
another disk, the second disk can be read and edited. 



TEST ASSEMBLY 

Test assembly offers a* wide set of construction choices. Items are 
selected by specifying content area and sub-classification. Cognitive level, 
user-defined catagories, item difficulty and item discrimination may also be 
used to choose sets of items. TAP then searches for itew fitting the 
characteristics selected. Each item fitting the selected characteristics is 
shown on the terminal in order for the user to accept or reject. Random 
selection is net available. 

A master copy and answer sheet are printed for each test. A set of 
instructions for the students is printed at the top of the first paqe. 
Individual items from the bank may be printed at any time. 



TEST ADMINISTRATION AND SCORING 

TAP has no provision for scoring or on-line administration. 



STUD BIT RECORDKEEPING 

There is no provision for student recordkeeping. 
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USE 

A comprehensive 102-page Manual comes with the program. It is detailed 
and easy to read. Several appendixes discuss special topics of item banking 
such as Bloom's Taxonomy of Cognitive Levels, determining item statistics, and 
writing multiple-choice questions. 

Menus guide the user through each stage of item entry and test 
development. If the user has a question, the manual can answer it. If the 
manual is not close at hand, a Help feature can be accessed. 

The major problem seen in using TAP was the problem of few items per disk 
(due mainly to not compressing items) and the associated problem of not being 
able to use multiple disks to construct a test. A second, sore minor, 
programing problem involves the time required to enter a large number of 
items. After each item is entered, it must immediately be saved to prevent 
loss of items. After saving an item, the computer returns to the nmin menu 
where the user must again request the item edit mode. 
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PRODUCT: TestBank 

VENDOR: Holt, Rhinehart, and Winston, Publishers 
383 Madison Ave. 
New York, NY 10017 



GHJERAL DESCRIPTION 

TestBank is an item banking and test construction program powerful enough 
for testing specialists, but easy enough for classroom use. Previously 
developed items may be used or new items may be added to the item bank. 

This package requires an Apple computer with at least 64k of memory and an 
80-column board, it is best suited for the lie or lie. TWO disk drives are 
required. Any printer available for the Apple computers can be used, but 
special characters are only supported on the Epson FX-8 and MX- 80, the 
Imagewriter, and the OKIDATA printers. 

No price was available at the time of this writing. The program comes 
with a backup disk, a comprehensive manual, and the specific item bank(s) 
ordered. 

Another program available from the publisher is Class II. It is 
apparently intended to be used with TestBank to handle some activities not 
covered by TestBank, such as computing test scores, maintaining student 
records, and calculating item statistics. Class II was not reviewed at this 
time. 



ITEM BANK DESCRIPTION 

Items are entered as one continuous question, stem and responses are 
entered together. This allows the "item" to be used as a passage or essay 
question. Passages may be up to 54 lines long, if they exceed the nmximum 
item length, a second (or third) consecutively numbered item may be used to 
continue the passage. Any item may be linked to a passage. When printing a 
test with a linked item, tne computer will first print the passage. 

Each item is stored with a four-digit objective number and an optional 
four-digit objective type. Up to ten items may be stored per objective number 
on a given disk. 

Item disks are password protected and hold between 200 and 1,000 items, 
depending on average question length. The fifth grade mathematics disk 
reviewed contained approximately 400 items and was about one-half full. 
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ITEM BANK MAINTENANCE 

Items are entered and edited using full-screen editing features. Line 
wrap, insert, delete, and full cursor movement are provided. Hard returns can 
be included at any place— and are needed in orter to locate the answer options 
properly. 

Users can use upper and lower case, underlining, and superscripts; as well 
as a long list of mathematical symbols including Pi and square root. Most of 
these characters are rarely seen in other programs. All special charaters are 
also mailable on the II* , despite its limited keyboard. 

TEST ASSEMBLY 

When constructing a test, the instructor has several options available. 
Tests can be composed of items selected randomly from the bank, or items can 
be selected from specified catagories. Selection within category can be 
either random or user-guided. Unlike many programs, items cannot be scanned 
as they *re selected. 

A master copy and answer sheet are printed for each test. If the user 
wishes a copy of items in the bank, they may be printed. Maintaining a hard 
copy of the item bank is necessary in order to select items for tes'-s. 

TEST ADMINISTRATION AND SCORING 

Testbank has no provision for scoring or on-line administration. Class 
II, a compatible program from Holt, Rhinehart, and Winston, uses TestBank item 
files to construct and score on-line tests. 

STUDENT RECORDKEEPING 

There are no provisions for student recordkeeping within this program. 
Again, they are available with Class II. 

USE 

A 58-page manual comes with the program and item disks in a hard case. 
The manual is detailed and easy to read. All aspects of the program are 
carefully described. 

TestBank uses its own disk operating system— increasing speed and security 
and providing the special character features. 

Fbr the most part, the program is easy to use, being menu driven. 
Initially some confusion can develop over the meaning of some words in the 
menus. Reading the manual is important in order to prevent confusion. Once 
the meanings are clear, the program runs quite well. 
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PRODUCT: TEST RITE 

AUTHOR: Richard Vanden Betg 

VENDOR: Class 1 Systems 

17909 Maple Street 
Lansing, IL 60438 



6QIERAL DESCRIPTION 

Test Rite is a program for creating and maintaining test item files. Items 
are stored in multiple-choice format, but may be printed in any format. Essay 
questions may also be stored. The package is excellent for regular classroom 
test construction. 

Test Rite uses the Apple II line of computers with 48R of memory, one or 
two disk drives, and virtually any printer. It is priced at $139 and comes 
with a well-organized, comprehensive manual including many example tests and a 
quick reference card of program menu options and commands. Versions are also 
available for TRS-80 and IBM PC computers. 



ITEM BANK DESCRIPTION 

Items are stored as an open-ended question and potential responses, the 
correct response entered first. Op to five responses per item are allowed. 
Each item (including responses) can have up to 243 characters, items my be 
given a four-character reference number for classification or other 
identification. Essay questions are stored in a separate file. The user does 
not need to follow the required format convention, but nonconformity could 
result in incorrectly worded test questions if care is not taken when 
selecting items. 



ITEM BANK MAINTENANCE 

Items may only be added or removed and given reference numbers. These 
reference numbers may be changed at any time. There is no full-screen 
editor. Editing is done by rewriting the entire stem or response. 



TEST ASSEMBLY 

Test Rite selects items for a test in a manner different from other 
similar programs which adds time, but saves disk space. The user must pass 
through the item bank file and identify which items are not to be included in 
the printed test. A second pass allows the instructor to seler w by 
classification from the remaining items. This second pass is completed by a 
series of greater -than and less -than commands which select from among the 
available reference numbers. If two items have the same reference number, 
they will both be selected. 
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Before printing, the user may add additional information to be printed on 
the test, such as the school name and course name and instructions up to 250 
characters long. Pre-written instructions are available for all types of 
questions, with variations for those answered on paper or on machine graded 
cards. 

Test item files are stored by both course name and course number. 



TEST ADMINISTRATION AND SCORING 

Test administration is by printer only. The printed copy is ready for use 

without further retyping. Question sets may be confined so that a long test 

using a variety of item kinds can be produced. An answer key is printed on a 
separate sheet. 

There is no provision for computer scoring. 



STUD BIT RECORDKEEPING 

There is no provision for student recordkeeping. 



Test Rite is entirely menu driven. The accompanying manual is complete 
and well organised. Anything not clear from the menus can be quickly 
understood, from the manual. The author even includes suggestions for 
developing tests, such as finding good dis tractors for multiple-choice items 
and procedures for computing split-half reliabilities. 

When choosing from menus, the user is saved the extra key-stroke of 
hitting "return" after each choice. Errors are caught by the program, but 
diagnostics are not provided — you can only try again. 
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PRODUCT: Testmaster 

AUTHOR: Michael W. Horner 

VENDOR: Midwest Software 
P.O. Box 214 
Farmington, MI 48024 



GENERAL DESCRIPTION 

Testmaster is a program for creating and maintaining item files and 
creating test files from those item files. This program is useful for 
producing regular tests in the classroom, saving much test compilation time. 
Alternate tests forms may be created to reduce classroom cheating. 

This package runs on Apple II, Commodore 64, or CBM computers. It is set 
up for one-drive systems and does not expand to two drives, it is priced at 
about $35 and includes one copy protected program disk and an 18-page nmnual 
in a convenient storage folder. 



ITEM BANK DESCRIPTION 

Items are stored in up to three files per disk, with up to 65 items per 
file. Thus up to 195 items may be stored on an **pple disk. Commodore users 
can store up to 80 items per file and 240 per disk. Piles are stored in 
reserved blocks on the disk, saving additional retrieval time by compressing 
edited files. 

Item stems can contain over six lines of material— much more than many 
similar programs, item type must be homogeneous within a file. Multiple 
choice, true/false, and completion questions may be used, but only one kind 
may be used in a given file. 



ITEM BANK MAINTENANCE 

A full-screen editor is provided to assist in adding and editing items. It 
is easy to use and provides a command line at the bottom of the page to assist 
in choosing the correct commands. 

Multiple choice questions may have between 2 and 5 responses. A 
true-false question is, in effect, a two-response multiple-choice question. 
Completion questions are stored as if they were one-response multiple-choice 
questions. Correct answers muat be entered first. This saves' the step of 
identifying the right response. When presented, the item responses are 
scrambled • 

After ten items are entered they are automatically saved, if the user 
wishes to save new items or changes more often, that can be done. 
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TOST ASSEMBLY 



ERIC 



Tests can be composed by randomly selecting a specified number of items 
from each of up to 10 item files, up to 100 items per test. Provision is 
given to select specific items. This is done by scrolling through the entire 
file selecting or rejecting each item one at a time, which can be time 
consuming in a large item file. 

Items are then randomly ordered with responses also randomized within ea<h 
question? providing for multiple copies of the same test at no extra effort. 

TEST ADMINISTRATION AND SCORING 

The primary use for Taskmaster is to produce printed tests. After 
assembling a test, the printed version is a random presentation of the 
selected items. Multiple randomized copies may be printed. No provision is 
provided for linking groups of items together. The program is designed to 
work with almost all available printers. 

When printing the test, the answer key and a student response sheet are 
also printed. The student response sheet is simply a sheet with room for the 
student's name and a series of numbers followed by blanks on which to write 
the answer. This is most useful for short answer questions because the test 
question sheet does not provide spaces between items. One shortcoming to the 
printing subroutine is that the user must stop after each page and advance the 
paper manually. 

STUD HIT RECORDKEEPING 

There is no provision for student recordkeeping. 

OSE 

Though the program is menu driven, some menu commands are not 
self-explanatory (e.g., "spool" means print item file). Thus the manual is 
important to keep handy. Unfortunately the manual is not organized in a clear 
and useful manner. One manual serves both Apple and Commodore users. In 
addition, it is written as one continuous section. If it were not for an 
index, finding information would require reading the entire 16-page manual. 

Once the commands are understood, the program is quick and easy to use. 
Owners of two-drive systems may be a bit frustrated at having to repeatedly 
switch disks. An item review feature is handy. It shows each item on the 
screen one at a time beginning with a given item. Presentation is automatic 
and the user can stop reviewing at any time or pause to edit. 

All errors encountered while reviewing this program where caught. In 
addition, after most menu choices, the user can opt out of the choice and 
return to the menu. This approach prevents being trapped into doing something 
not desired. 
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PRODUCT: Testmaster Series 



VENDOR: D.E.C. Computing 

5307 Lynnwood Drive 
West Lafayette, IN 47906 



GBiERAL DESCRIPTION 

The Testmaster Series is a series of item banks in algebra, geometry, and 

chemistry at the high school level. They are intended to be used with a word 

processor package such as "Wordstar" or "EasyWr iter". The word processor is 

used to edit questions and add new items, printing out the final test. 

Item sets are available for schools with the IBM PC or PCjr and cost about 
320 MCh. 

No copy of the item bank was available for review. 

D.E.C. Computing also has a program called "Gradebook" available for 
TRS-80, EJM, and Apple computers, it can handle up to 60 grades for up to 400 
students. Grades are kept for individuals and classes. Grades may be 
weighted according to any formula. It sells for about $35. 
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PRODUCT: TESTS, CA ICR EAT, CAITAKE 

AUTHOR: Jerome Thayer 

VENDOR: Edu Systems 

3224 Lakeland Drive 
Berrien Springs, MI 49103 



GBJERAL DESCRIPTION 

TESTS is an item authoring and test construction program for classroom 
use. CAICREAT and CAITAKE are programs for creating and administering on-line 
quizzes and tutorials. 

These packages are available for the IBM PC or compatible computer with 
128K memory and one disk drive. TESTS is available alone for $75. All three 
programs sell as one package for $100. 



ITEM BANK DESCRIPTION 

TESTS creates item files limited in size only by disk storage limits. 
Each item can have up to fifty 70-coluim lines for the stem and all 
responses. This allows room for passages or graphics within the item. A 
variety of graphics symbols and special characters can be embedded in an item 
to display on the screen or print. The program automatically creates a second 
file which holds a list of objectives which are linked to each item. These 
objectives can be of any form desired by the user. 



ITEM BANK MAINTENANCE 

Item files may be created or modified at any time. A full-screen editor 
is provided with its commands readily visible on the screen for quick 
reference. The major shortcoming of the editor is that there is no 
wrap-around. This problem is e? *y to adjust for since the test questions 
print exactly as viewed. 

Items may be copied to other files and parts of files can be merged to 
form new tests as desired. 

Correct answers are stored in a separate file from the items. While 
entering the answers, the user cannot see the item (unless it was first 
pr in ted). 

When editing items in an existing file, the user must either remember the 
exact name of the item storage file or ask for a list of files. Row to get 
that list is not raadily apparent, but easy to do once learned. 
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TEST ASSEMBLY 



Actual test construction is done at the time of printing or presentation. 
Before printing, the instructor makes an item selection file using TESTS. 
This file identifies which objectives and items within objectives are to be 
used. The instructor car. also choose items randomly within objective. 

When choosing items or labeling categories, a block search feature is very 
useful. This feature allows the user to look for any word or part of a word 
in all items in the file. Thus if the instructor -ants to find all the items 
in a history file which mention George Washington, the computer can be used to 
search for the string "Washington". 



TEST ADMINISTRATION AND SCORING 

CAICREAT constructs command files used by CAITAKE to give the test 
previously constructed with TESTS on the computer, either as a test or a 
tutorial. CAITAKE could not be evaluated. An apparent bug in the program or 
a damage to the disk prevented progress within the program past the first 
decision point. 

According to the manual, students are given items in the order th*y are 
stored within each objective, wi;-ii a choice after each item to continue in 
that objective or to go to a new one. Students are given feedback after erch 
response. This feedback is very flexible and is designed by the instructor. 
As items are presented, a tally is kept on the screen of the number and 
percentage of items on the current objective which have been answered 
correctly on the first attest 

Printing tests with TEST* is very easy. Before printing, the file can be 
amended to provide ready- to-nse tests—with one exception. In order to use 
items with CAITAKE the first letter of the first line in the item vast be the 
answer of tne multiple-choice question. This letter will be printed, and Uius 
must be removed before copying. 



STUDENT RECORDKEEPING 

No mention is made in the manual of storing student records. 



USE 

The set of three programs comes with a 15-page manual which adequately 
covers the information needed to use the programs. The programs, though menu 
driven, require keeping the manual close at hand. Though the package provides 
a variety of features not generally foun<3 in similar programs, the user must 
be ouite familiar with the operation of the IBM-PC. Some familiarity with 
PC-DOS is necessary. Users with the most experience with the disk operating 
system will be able to make the most use out of the program, In addition, 
several apparent bugs still exist. 
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PRODUCT : Tests Made Easy 

AUTHOR: Don Kraf check 

VENDOR : Compu-ta tion s 
P.O. Box 502 
Troy, MI 48099 



GBiERAL DESCRIPTION 

Tests Made Easy is an item storage and test construction program. Tests 
are produced by printer for classroom use. There is no provision for on-line 
presentation. Questions may be chosen randomly or by number. 

The program runs on the Apple II computer with either one or two disk 
drives and sells for about $30. 



ITEM BANK DESCRIPTION 

Question files of up to 300 items can be entered and saved. This program 
is designed to handle questions of the short answer, completion, or essay 
variety. Questions may have up to 600 characters. Answers may be stored with 
the questions. These, too, may have up to 600 characters. The user may also 
specify the number of blank lines to follow the question when printed. Thus, 
essay and short answer questions may be included in the same test without 
having to retype the test. 



ITQ4 BANK MAINTENANCE 

No editor is provided with the package. This means that even long 
questions must be typed perfectly or be entirely retyped. In addition, no 
cornas or colons w*y be used in the question. 



TEST ASSEMBLY 

Tests may be assembled from an item f lo randomly or by selecting specific 
items. When selecting specific items, the user may designate item numbers or 
scroll forward or backward through the file to identify questions. 

Multiple forms may be printed by forming an item file with the exact items 
desired and then twice selecting all of the items randomly. This provides two 
tests with different item order. 



TEST ADMINISTRATION AND SCORING 

hV tests are administered manually. Printing is rather slew. There is 
no provision for on-line presentation. 
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STUDENT RECORDKEEPING 



No provision is made for storage of student records. 



USE 

The program uses an easily followed menu format. A short manual explains 
most that the user needs to know. Por what it does. Tests Made Easy is easy 
to use—though the printing is slow and there is some delay in file 
retrieval. 
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PRODUCT: 



Tes tWorks 



AUTHORS: 



Mark Sweetnam, William Knight, and Philip Roth, Jr. 



VENDOR: 



Milliken Publishing Co. 
1100 Research Blvd. 
P. O. Box 21579 
St. Louis, MO 63132 



GENERAL DESCRIPTION 

TestWorks is a program for creating and maintaining item bank files. It 
can be used to print regular classroom tests. On-line use include producing 
regular tests and minimum competency tests or to aid in nstery learning by 
allowing students to review their answers. 

TestWorks is available for any Apple computer with a minimum of 64R 
memory. Either one or two disk drives may be used. The program sells for 
about $250 and includes a manual, an instructor disk and backup, a blank item 
disk, a classroom disk, a help disk, and a tutorial disk; all in an 8 1/2" by 
11" binder. Extra disks and material may be purchased for $10 to $60. 



ITEM BANK DESCRIPTION 

Items are stored in multiple-choice format. Up to five responses are 
allowed and each item may have up to five lines in the question. Bach item 
bank disk can hold up to 300 items, A code (digits or letters) of up to 14 
characters can be given to each item. With this code, teachers can classify 
items with minute detail Each item is also stored with an estimated 
difficulty rating. These ratings are on a scale of 0.2 to 5.0. 



ITEM BANK MAINTENANCE 

A minor shortcoming of Testtforks is the item editor, though actually 
having an editor puts this program above several others reviewed. Item lines 
do not wrap around when entered. If several words need to be deleted, the 
rest of the item should be retyped in order to maintain the question's 
appearance and readability. 

Many programs with word wrap use the hard re' <rn to end entry. With 
TestWorks hard returns simply move the cursor, inis allows construction oi 
items unavailable with other programs, such as mathematics problems in a 
vertical format. 

The strongest aspect of TestWorks is the system built in to maintain item 
records. For each item, the computer records the number of times that each 
item was presented in an on-line test and how often each response was given. 
In addition, the item difficulty estimate is adjusted with each use. This 
adjustment may be turned off; based simply on the percent of correct 
responses; based on percent correct and the total score of each student; or be 
based on percent correct, total score, and an adjustment for guessing. 
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Records are also maintained for each defined test. The user may obtain 
the mean, the standard deviation, and the minimum and maximum test scores 
obtained to date. The computer also lists the name of the student with the 
highest ran and adjusted scores. A separate listing provides the same 
statistics for the difficulty ratings of the items in the item file. 



TEST ASSEMBLY 

Tests may be selected in almost any manner imaginable. The instructor may 
select specific items from the file to be used on a test. The computer can be 
told to select randomly from the file. Selection can also be based on a range 
of item difficulties or on item classification codes (or portions of the 
codes). All of these approaches to test construction may be used in 
combination. If desired, a teacher may identify a few items that definately 
will go on the test, and then allow the computer to choose randomly from a set 
of items identified by difficulty and code. Parallel tests may also be 
constructed. In this case the computer randomly selects items in such a way 
that the average difficulty of the test is equal to the average difficulty of 
the file. Tests can also be sorted by item difficulty so that the test gets 
harder as the student answers more questions. Item difficulty can also be 
used to create parallel tests using different items of similar difficulty. 



TEST ADMINISTRATION, SCORING, AND RECORDKEEPING 

Tests may be presented by paper or directly on the computer. When using 
the computer, the student uses a classroom disk and the item file disk. These 
are very secure. Given just these two disks, a student can take a test 
virtually unsupervised. If the school has several Apple computers, copies of 
the appropriate disks may be made and the test given on more than one computer 
at a time. If a student removes a disk or turns the computer off before 
completing a test , the disk is "locked" and can only be used again after the 
teacher uses the main program disk to unlock it. This discourages cheating 
and tells the instructor that a student failed to complete the test. 

When taking a test at the computer, the student has the option, at the 
teacher 1 s discretion, to skip items. Skipped items my be answered when the 
other items are completed or the student may choose to not answer those 
questions. Teachers may also choose to use a test as a learning experience. 
In such instances, the stude _s may review their incorrect answers after 
completing the test and learn from their mistakes. Instructors my also allow 
the computer to give immediate feedback after each question. This feedback 
can be changed at any time. 

Printed tests include a separate answer key for the teacher's use. Tests 
presented by computer are automatically scored. After taking a test, each 
student is immediately presented with their score. This is stored on the 
classroom disk for the teacher to access later. In addition, descriptive 
statistics are maintained on the entire class. 
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Tests are scored in six ways. Three ways are based on the total number of 
items in the test (absolute score) ; the other three ways are based on the 
number of questions answered (relative score). These two sets of scores only 
differ if students are allowed to, and choose to, skip items. Absolute and 
relative scores are given in terms of (1) the raw score, (2) the adjusted 
score, and (3) the weighted score. The raw score is just as it sounds— simply 
the percent correct. The adjusted score is only used when parallel tests are 
used; tt is based on the average item difficulty and helps adjust for slight 
differences between test forms. Lastly, the weighted score is based on the 
difficulty of the items answered correctly. High weighted scores can point 
out good students who answered carelessly (getting hard items right and easy 
items wrong). If the entire class has high weighted scores, it may indicate 
that the initial estimate of item difficulty was incorrect and needs to be 
adjusted. 

Tests presented on paper can still be included in the item statistics and 
student records, in such situations the instructor must enter each student's 
answers by hand. This can be time consuming, but does allow for updating item 
difficulties and maintaining records. Some of the diagnostic aspects of the 
TestWorks score reporting could make hand entry worthwhile. 



OSB 

A 51-page manual accompanies the various program disks. It is clearly 
written and well organized. Each aspect of TestWorks is laid out in a step-by 
step manne- «*inus guide the user through each step. If at any time the user 
is unclea, ; what a menu item refers to and does not want to look it up in 
the manual, « aeparate help disk is available. It can be accessed from each 
menu, at which time only information on that menu will be given, in all, 
TestWorks is easy to use and quick to produce tests. It does not have all the 
features of a program like MicroCAT, but it does produce usable tests with the 
additional feature of item statistics at a reasonable price. 



PRODUCT: THE SAGE 



AUTHOR: Paul Cozens 

VENDOR: Jagd staff el Software 
645 Br ends lee Drive 
San Jose, CA 95123 



GENERAL DESCRIPTION 

The Sage is a program for creating and maintaining files of tests. It is 
intended for use "in home, school, or industry* , but is most practical in the 
hone or elementary classroom. 

This program is written for an Apple 11+ or He computer with 48K of 
memory. Two disk drives are required. 

It can be purchased for about $62 and comes with a program disk, a work 
disk and a 52-page manual. 



ITEM BANK DESCRIPTION 

Dp to eight tests of up to 25 questions each may be stored on a disk. 
Items can be written in many forms (multiple-choice, true/false, completion) . 
Items are entered with (1) stem, (2) response option* in one list, (3) the 
number of correct responses possible, and (4) the correct responses. The stem 
is limited to six lirss. Responses may be very lengthy, but when entering 
them the user must pad with blanks instead of hitting •return* if one response 
per line is desired. There is no provision to provide wltiple correct 
responses to a single completion response, if, for example, the correct 
response to a question is •two 11 , the user cannot allow for a student who types 
in m i m . The computer will count it as wrong. Completion items may have more 
than one blank, and thus the question may have several separate answers. 



I TO! BANK MAINTENANCE 

Items may be added or deleted at any time. 

There is no editor included in the package. If an item includes an error, 
it must be entirely reentered. The return key is used to terminate entry of 
stem and answer. Questions print as shown on the screen, so each line must be 
padded with blanks in order to read correctly. 
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TEST ASSEMBLY 



Item files are tests. Purther selection ^3 not provided, if an item 
stored in a test is not wanted, it must be deleted before presenting to the 
students. A provision is allowed to copy a test— but it is complicated and 
not clearly described in the manual or on-line menus. A copied test could 
have items deleted and still allow for those deleted items to be stored on the 
disk—in the original copy. 



TEST ADMINISTRATION AND SCORING 

The Sage is designed for on-line administration and scoring. its best 
use is for review and practice. Any student using the program can choose 
between "studying" a test or "taking" a test. Thus they can see the correct 
answers before actually taking the test. The program does record each 
students score on every attempt at each test—with separate records for 
review mode and test mode. Thus an instructor can keep track of each 
student's activities with The Sage and how well they did with and without 
reviewing the answers. 

Since questions may have more than one answer, two scoring methods are 
available, (toe method gives each question equal weight in computing the 
total, with this first method a ten-question test would give ten points to 
each question whether there was one answer or five in any question. The 
second method allots equal points to each answer. In a ten question test with 
a total of twenty answers, each answer would be worth five points. These 
options are most worthwhile when the test contains completion questions with 
more than one blank. 

Test items can be printed, but the printed copy includes the answers. 
Printing tests are useful for saving a hard copy, but not for class 
presentation. 



STUDENT RECORDKEEPING 

Student records are maintained of the roost recent scores on each test. Dp 
to 50 student records are allotted. Each student has access to only their own 
record and cannot change it. The teacher can view all records, as well as 
performing initial set-u~ jni file cleaning. The teacher gains access by a 
password that is fu*\ visible when entered—threatening its security, though 
it can be changer" 
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USE 

The manual accompanying The Sage is well organized, but written as a 
tutorial. Finding specific information in it is difficult. It is unclear on 
some basic topics such as how questions are formatted and scored— making 
initial use especially confusing. One section describing the complicated 
procedures for coding a test was actually incorrect. 

Item entry is awkward since words are broken at the end of the line and 
not at the end of the word. The user has to type in the question with padded 
blanks in each line in order to make the question readable, since a return 
will terminate entry. 

Much of the program contains unnecessary repetition. Hany routines end by 
putting you back to the initial log -on instead of the main menu. Before you 
are shown a menu, you must type in the code for "show menu". This is a waste 
of time and program length. Whenever a new question is added to a test, the 
user mist again declare the scoring procedure. This should be in a separate 
routine. 
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